MDEV-40827: prefetch MHNSW neighbours during search - #5571
Conversation
Issue prefetches for unseen neighbour nodes before evaluating their distances. This overlaps later memory loads with distance calculations for earlier lanes without changing the search result. On a fixed 200k by 1024-dimensional cosine graph with M=6, a crossed AB/BA warm-cache run (30 paired observations) improved paired median QPS by 10.1% at ef_search=40 (95% CI 8.0%-14.4%) and 13.8% at ef_search=160 (95% CI 11.8%-20.0%), with identical exact-recall means.
|
|
gkodinov
added
the
External Contribution
gkodinov
left a comment
gkodinov
left a comment
There was a problem hiding this comment.
Thank you for your contribution! This is a preliminary review.
Please consider increasing the compiler coverage as suggested below. And stand by for the final review.
| if (res == 0xff) | ||
| continue; | ||
|
|
||
| #if defined(__GNUC__) |
There was a problem hiding this comment.
please enable this for clang and msvc. This is what grok says should be done:
#if defined(__GNUC__) || defined(__clang__)
__builtin_prefetch(link, 0, 3); // rw: 0=read, 1=write; locality: 0–3
#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64))
#include <xmmintrin.h> // or <intrin.h>
_mm_prefetch((const char*)link, _MM_HINT_T0); // _MM_HINT_T0, _MM_HINT_T1, _MM_HINT_T2, _MM_HINT_NTA
#else
// no-op or do nothing
(void)addr;
#endif
There was a problem hiding this comment.
There is also PreFetchCacheLine , which can be used just with _WIN32 , i.e does not depend on architecture
There was a problem hiding this comment.
There is also PreFetchCacheLine , which can be used just with _WIN32 , i.e does not depend on architecture
Good idea, added to make prefetch portable!
a8075c1 to
edf9a46
Compare
|
Addressed in the latest commit. Now supports GCC/Clang, MSVC x86/x64, and MSVC ARM64. |
a651451 to
c8da29b
Compare
c8da29b to
7868523
Compare
There was a problem hiding this comment.
Pull request overview
Adds portable CPU cache prefetching for unseen MHNSW neighbors during vector search.
Changes:
- Prefetch neighbor allocations before distance evaluation.
- Add cross-platform
my_prefetch()support.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
sql/vector_mhnsw.cc |
Prefetches unseen MHNSW neighbors during search. |
include/my_cpu.h |
Provides portable prefetch helper implementations. |
Suppressed comments (1)
sql/vector_mhnsw.cc:1360
- This assumes that the vector begins in the cache line immediately after the node's line, but the allocation is
FVectorNode + gref + tref + vector, and the supported reference length can be much larger than one cache line. For such indexes this prefetch only warms the node/reference area;distance_to()still starts on a cold vector, so the optimization misses its target. Prefetch the actual vector address (usinglink->vecwhen loaded, otherwise derive it fromtref() + tref_len()withFVector::align_ptr()).
my_prefetch(link);
my_prefetch(reinterpret_cast<const char*>(link)
+ CPU_LEVEL1_DCACHE_LINESIZE);
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Summary
Prefetch unseen MHNSW neighbour allocations before their distance evaluations.
Correctness
#elsepath also compiles.--do-test=vectorandmain.mysqld--helppass in both.Performance
Counterbalanced AB/BA, warm cache, both binaries querying the same persisted graph (200k x 1024, cosine, M=6); 200 queries, 30 paired observations per cell.
AMD EPYC 7713, RelWithDebInfo, 8-CPU cgroup quota on a shared host.
ef_searchThe gain grows with
ef_search: largerefexpands more nodes, so more neighbour groups are scanned and more memory latency is available to hide.